This invention relates to a motion vector detection used in the motion-compensated predictive coding known as a technique for compressing a digital moving picture, and more particularly to a motion vector detection circuit for implementing sampling to an input image and a comparative image at multi-stages to carry out motion vector detection in a hierarchical manner.
As the conventional hierarchical motion vector detection technique, there is known a technique shown in FIGS. 1 and 2. The principle of the hierarchical motion vector detection technique is shown in FIG. 1, and the block of a hierarchical image is shown in FIG. 2. Explanation will now be given by taking an example of the case where an image subject to processing is constituted by 512.times.512 pixels to carry out motion vector detection at three stages using a block constituted by 8.times.8 pixels as a unit.
As shown in FIG. 1A, an input image N3 is subjected to 1/2 subsampling in both horizontal and vertical directions, resulting in an input image N2. The input image N3 is also subjected to 1/4 subsampling in both horizontal and vertical directions, resulting in an input image N1. Namely, 8.times.8 pixels of the input image N1 at the first stage correspond to 16.times.16 pixels of the input image N2 at the second stage, and further correspond to 32.times.32 pixels of the input image N3 at the third stage.
On the other hand, as shown in FIG. 1B, similar subsampling is implemented also to the comparative image C3 for detecting the motion vector. Namely, a comparative image C3 is subjected to 1/2 sampling in both horizontal and vertical directions, resulting in a comparative image C2. The comparative image C3 is also subjected to 1/4 sampling, resulting in a comparative image C1. Namely, 8.times.8 pixels of the comparative image C1 at the first stage correspond to 16.times.16 pixels of the comparative image C2 at the second stage, and further correspond to 32.times.32 pixels of the comparative image C3 at the third stage.
Initially, in the motion vector detection at the first stage, the input image N1 and the comparative image C1 are used. The input image N1 is divided into blocks of 8.times.8 pixels. Then, block matching using full search method is carried out every block in a search region of .+-.7 pixels in both horizontal and vertical directions of corresponding blocks on the comparative image C1.
Subsequently, in the motion vector detection at the second stage, the input image N2 is similarly divided into blocks of 8.times.8 pixels. Then, a motion vector having the length twice larger than that of three motion vectors in total of a corresponding block of the input image N1 at the first stage and adjacent blocks thereof is used to carry out matching processing. As a result, a search in a range of .+-.1 pixel in both horizontal and vertical directions of a motion vector in which an error becomes minimum is carried out. Thus, motion vectors at the second stage are detected.
For example, as shown in FIG. 2, processing blocks at the second stage are assumed as a, b, c and d. Further, a block at the first stage corresponding to these blocks a, b, c and d is assumed as Bo. In this case, as the adjacent blocks, motion vectors of the blocks B.sub.1 and B.sub.2 are used with respect to the block a; motion vectors of the blocks B.sub.1 and B.sub.3 are used with respect to the block b; motion vectors of the blocks B.sub.2 and B.sub.4 are used with respect to the block c; and motion vectors of the blocks B.sub.3 and B.sub.4 are used with respect to the block d.
Similarly, at the third stage, the input image N3 is divided into blocks of 8.times.8 pixels, and similar processing is implemented to these divided blocks. Thus, detection of motion vectors at the three stage is carried out.
The above-mentioned motion vector detection processing is carried out by using, e.g., a circuit shown in FIG. 3. In this figure, a comparative image C3 is stored into a frame memory 10 on one hand, and is subjected to 1/2 subsampling in both horizontal and vertical directions at a subsample circuit 12 on the other hand. A comparative image C2 generated by this subsampling is stored into a frame memory 14 on one hand, and is subjected to 1/2 subsampling in both horizontal and vertical directions at a subsample circuit 16 on the other hand. A comparative image C1 generated by this subsampling is stored into a frame memory 18.
On the contrary, an input image N3 is stored into a frame memory 20 one one hand, and is subjected to 1/2 subsampling in both horizontal and vertical directions at a subsample circuit 22 on the other. An input image N2 generated by this subsampling is stored into a frame memory 24 on one hand, and is subjected to 1/2 subsampling in both horizontal and vertical directions at a subsample circuit 26 on the other hand. An input image N1 generated by this subsampling is stored into a frame memory 28.
Image data stored into respective frame memory groups 30 and 32 in a manner stated above are suitably read out by a motion vector detection circuit 34. Detected results are stored into memories 36, 38 and 40, respectively. Thus, a final motion vector output is provided from the memory 40.
The detail of the above-described motion vector detection circuit 34 is shown in FIG. 4. The operation of this circuit is as follows.
a. Motion vector detection at the first stage
In this case, on the input image side, one block of 8.times.8 pixels is read out from the frame memory 28 by means of a block readout circuit 42, and is stored into a work memory 44.
As described above, the search range at the first stage is .+-.7 pixels in both horizontal and vertical directions. For this reason, on the comparative image side, corresponding data of 22.times.22 pixels is read out from the frame memory 18 by means of a block readout circuit 46, and is stored into a work memory 48.
At this time, pixel data newly delivered from the frame memory 18 to the work memory 48 has a data size of 15.times.22 pixels as shown in FIG. 5A in the case where a block subject to detection is the block at the left edge of the input image N1. Further, that pixel data has a size of 8.times.22 pixels as shown in FIG. 5B in the case where the block subject to detection is another block than the above.
Then, on the input image side, data of blocks stored in the work memory 44 are read out per each pixel by means of a pixel readout circuit 50. On the other hand, on the comparative image side, data of blocks corresponding to 225 trial vectors are sequentially read out from the work memory 48 per each pixel by means of a block selection circuit 52. In other words, in the search range of .+-.7 pixels centering the block of 8.times.8 pixels, there exist 15.times.15 blocks, and there exist trial vectors for respective blocks. Thus, data are sequentially read out for respective trial vectors, and are compared with those on the input image side.
Respective readout data by the pixel readout circuit 50 and the block selection circuit 52 are all delivered to a matching circuit 54. In the matching circuit 54, errors between pixels, e.g., mean square error (MSE) or mean absolute error (MAD) are calculated for 225 trial vectors respectively. Errors thus calculated and trial vectors are delivered to a vector judgment circuit 56.
In the vector judgment circuit 56, comparisons among the 225 errors sequentially inputted are carried out, and the minimum value of the errors and a trial vector corresponding thereto are found out. The trial vector thus found is written into the memory 36 as a motion vector at the first stage.
The above-mentioned operation is carried out every block included in the input image N1 at the first stage. Thus, motion vectors are determined with respect to such blocks, respectively.
b. Motion vector detection at the second stage
In this case, initially, on the input image side, one block of 8.times.8 pixels is read out from the frame memory 24 by means of the block readout circuit 42, and is stored into the work memory 44.
On the other hand, on the comparative image side, three motion vectors in total of a motion vector of a block at the first stage corresponding to the block read out at the second stage and motion vectors of two blocks adjacent thereto (see FIG. 2) are read out from the memory 36 by means of the block readout circuit 46. Then, blocks of 10.times.10 pixels corresponding to vectors twice larger than these motion vectors at the first stage are read out from the frame memory 14 by means of the block readoutcircuit 42 for the three motion vectors at the first stage, and are written into the work memory 48 (see FIG. 6).
Then, a block of the block center 8.times.8 pixels shown in FIG. 7 for each of the three blocks of 10.times.10 pixels written into the work memory 48 in a manner stated above and a motion vector corresponding thereto are delivered to the matching circuit 54.
To the matching circuit 54, input image side data of blocks of 8.times.8 pixels are inputted from the work memory 44. Thus, the matching processing therebetween is carried out. As a result, a block in which an error becomes minimum and a motion vector thereof are determined by means of the vector judgment circuit 56. Thus, a number of the motion vector shown in FIG. 6 is delivered to the block readout circuit 46.
Data of 10.times.10 pixels in the work memory 40 corresponding to the determined motion vector is used to perform the operation similar to that at the abovedescribed first stage in the search range of .+-.1 pixel in horizontal and vertical directions except for the center. Namely, matching between the blocks in the search range and the blocks stored in the work memory 44 is carried out at the matching circuit 54. Thus, errors and motion vectors are delivered to the vector judgment circuit 56. Then, eight comparisons with the minimum error previously detected are carried out at the vector judgment circuit 56. Thus, detection of the minimum error and the motion vector detection at the second stage corresponding thereto are carried out. This motion vector at the second stage is written into the memory 38.
The above-mentioned operation is carried out for respective blocks included in the input image N2 at the second stage. Thus, motion vectors are determined with respect to respective blocks.
c. Motion vector detection at the third stage
In the same manner as that at the second stage, motion vector detection at the third stage is carried out by using detected motion vectors at the second stage and respective images N3 and C3 at the third stage. The detected motion vectors at the third stage are written into the memory 40. Thus, motion vectors between the input image N3 and the comparative image C3 are finally detected.
In the above-mentioned respective operations, the count operation of respective stages is carried out by means of a stage block counter 58.
With the conventional motion vector detection technique as described above, however, there exists an inconvenience described below. Assuming now that the number of pixels of initial images N3 and C3 is 512.times. 512, the number of blocks of 8.times.8 pixels is 64.times.64. Further, the number of pixels of images N2 and C2 at the second stage is 256.times.256, and the number of blocks of 8 .times.8 pixels is 32.times.32. In addition, the number of pixels of images N1 and C1 at the first stage is 128.times.128, and the number of blocks of 8.times.8 pixels is 16.times.16. For this reason, the number of pixels read out from the frame memory 18 at the first stage is expressed as follows: EQU 15.times.22.times.16+8.times.22.times.16.times.15=47520
Further, the number of pixels read out from the frame memory 14 at the second stage is expressed as follows: EQU 10.times.10.times.3.times.32.times.32=307200
In addition, the number of pixels read out from the memory 10 at the third stage is expressed as follows: EQU 10.times.10.times.3.times.64.times.64=1228800
The sum total of the above-mentioned number of pixels is equal to 1583520.
On the other hand, the number of one frame is expressed as follows: EQU 512.times.512=262144
As is clear from the comparison therebetween, the number of pixels read out at the time of vector detection is about six times or more than the value of the sampling frequency. For this reason, the prior art has the inconvenience that it is necessary to provide an access to the frame memory at a high speed.